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PREFACE 


This Standard has been prepared as an aid to scientists, engineers and managers who have responsi- 
bility for identification and interpretation of trends in NASA manned flight programs. The ocu- 
ment has been coordinated with Kennedy Space Center, Marshall Space Flight Center, and Johnson 
Space Center. Although written expressly for manned programs, the techniques covered may e 
applicable to unmanned programs. This document will be revised in the future as NASA s trend 
analysis efforts mature and expand. 


Data and examples in this document have been developed to aid in illustrating analytic techniques 
and should not be construed to represent actual data or trends. 


Questions should be addressed to the NASA Data Systems/Trend Analysis Division (Code QD), 
Office of the Associate Administrator for Safety, Reliability, Maintainability and Quality Assur- 
ance, Washington, D.C. 20546. 
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Chapter 1 

PURPOSE AND SCOPE 


This Standard presents descriptive and analytical techniques for NASA trend analysis applications. 
Trend analysis is applicable in all organizational elements of NASA connected with, or supporting, 
developmental/operational programs. Use of this Standard is not mandatory; however, it should be 
consulted for any data analysis activity requiring the identification or interpretation of trends. 

Trend Analysis is neither a precise term nor a circumscribed methodology, but rather connotes, 
generally, quantitative analysis of time-series data. For NASA activities, the appropriate and appli- 
cable techniques include descriptive and graphical statistics, and the fitting or modeling of data by 
linear, quadratic, and exponential models. Usually, but not always, the data is time-series in nature. 
Concepts such as autocorrelation and techniques such as Box-Jenkins time-series analysis would 
only rarely apply and are not included in this Standard. 

The document presents the basic ideas needed for qualitative and quantitative assessment of trends, 
together with relevant examples. A list of references provides additional sources of information. 


1-1 




Chapter 2 
BACKGROUND 


2.1 REPORTS BY ROGERS COMMISSION AND SLAY COMMISSION 

The Rogers Commission was appointed by the President to investigate the Challenger Space Shuttle 
accident The Commission’s mandate was to determine the probable cause(s) of the accident, and, 
based on its investigative findings, to provide recommendations for improvements ‘n ^ce program 
safety. The committee’s consensus was that sealing failure of O-nngs m the Shuttle s ngh t Solid 
Rocket Booster was the specific failure leading to the accident. Investigation of the circumstances 
surrounding the accident revealed that data on O-ring anomalies in previous shuttle flights showed a 
hazardous trend beginning in January 1984. However, no quantitative trend analysis was conducted 
prior to mission 51-L to reveal and substantiate this trend. Thus, the developing trend in O-nng 
erosion/blowby went unnoticed. 

In response to the Committee recommendations, the Office of Safety Reliability, Maintainability 
and Quality Assurance was established. The Office functions in a policy determination and over- 
sight capacity for all NASA SRM&QA activities, and has responsibility for ensuring that trend 
analysis techniques are applied, where appropriate, in NASA programs. 

In September of 1986, the committee on Shuttle Criticality Review and Hazard Analysis Audit, 
known as the Slay Commission, was established in response to a recommendation by the Rogers 
Commission. The Comma s purpose was to evaluaie NASA efforts m revrewmg safe 
shuttle components and identifying components needing improvement. The CommRt^ s report was 
more broadly focused, evaluating the entirety of STS risk assessment and risk management proce- 
dures. 

The Slay Commission recommended that NASA employ more quantitative methods and measuresin 
assessing STS risks, and in supporting NASA management decisions regarding the STS Trend 
analysis is one quantitative technique to identify potentially hazardous conditions based on pa 

empirical data. 

2.2 RELEVANT NASA GUIDANCE 

NASA Management Instructions 8070.3 and 8070.4 pertain to specific trend analysis requirements 
and risk management policies, respectively. NASA Handbook 8070.5 is a companion dement to 
NM1 8070.3 and further defines the responsibilities, classifications, and overall guidelines for the 
conduct of trend analysis for manned spacecraft and associated payloads. 

2.3 MOTIVATION FOR TREND ANALYSIS - BASIC OBJECTIVE AND BENEFITS 

In assessing system (or component) reliability and safety, the most complete quantitative knowledge 
woufd be in a reliability function (or equivalently, a risk function). The reliability of a system (or 
component) is a real-valued function with range in the interval [0,1]; it is a probability function, 
giving the probability of adequate performance under specified conditions up to a time, t Usually 
fAis reliability function, denoted R, is a function of a time variable and certain parameters. For 
example, a wear-out time-to-failure model widely used is the Weibull model R - exp(-at ), where a 
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and b are parameters to be estimated. More generally, sometimes the parameters of the reliability 
function are themselves functions of other variables, such as pressure and temperature. 

Determining the reliability distribution and estimating its parameters may be very difficult, costly, 
and perhaps intractable. Trend analysis is an alternative or companion approach. Specifically, we 
know that the values of certain variables will directly impact on a component or system’s reliability, 
even though the exact quantitative relationship or risk has not been determined. Those measurable 
variables (parameters) that directly affect system or component reliability are sampled over time. 
The variable values are examined to see if there is a pattern of deviation over time (i.e., a trend) 
from acceptable performance limits. In this manner, one may be able to predict future parameter 
values, or at least estimate the long-term range of values of these influential variables. In turn, if 
these parameters are trending towards hazardous or unacceptable levels, the potential problem could 
be identified prior to the occurrence of high-risk situations. 

The preceding trending approach is termed performance trending. In other applications, the focus 
may be less on detecting a deviation from an acceptable limit than on detecting an overall upward 
or downward movement of a set of observed values over time. This general trending could apply to 
programmatic or reliability concerns: for example, significant problem reports; workmanship de- 
fects; cannibalizations; relative risks and failure rates. Trending applied to management indicators is 
termed programmatic trending. When applied to frequency of occurrence, it is termed problem 
trending. In both of these domains, a trend analysis can reveal a movement towards unacceptable, 
undesirable or dangerous reliability, safety, or quality assurance levels. Also, if the particular 
trending model has a significant quantitative fit (e.g., linear, quadratic, exponential), future predic- 
tions can be made. 

From a management or reliability point of view, there are cases where a distinct trend should exist. 
For example, there should be an increasing trend in reliability with successive design changes and a 
decreasing trend in open significant problem reports as time-to-launch decreases. 

Thus, measurable variables will impact, directly or indirectly, system reliability, safety, and quality 
assurance. Even though the exact or approximate quantitative relationship to reliability may not be 
known, following the trending of these variables may identify potentially significant reliability/ 
safety problems. This proactive posture is both the objective and tangible benefit of the trend 
analysis approach. 
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Chapter 3 

TREND ANALYSIS - OVERVIEW 


3.1 THREE CLASSES OF TREND TYPES 

Chanter 2 of NASA Handbook 8070.5 provides the definitions for the three types of Trend 
Analysis: Performance Problem, and Programmer. The following summary the* defmmons. 

a Performance Trending. This trend analysis is used to measure a condition(s) that is 
' changing over time in a manner that eventually will cause the part/system to fatl. This 
techniaue can be considered a simple reliability model. A simplified example of a perfo 
mance trend would be the decline in electrical output of a deployed solar panel over time. 

h Problem Trending. This trend analysis tracks and categorizes problems ov e r time. The 
problems may be for an entire system, subsystem, or any other appropriate level of 
aggregation. This technique allows managers to focus on the problems that are occurn g 

with the most frequency. 

_ Programmatic Trending. This trend analysis quantitatively monitors program-related mdt- 
Z ZZt over, ml critical schedule elements, and manpower resource avatlabth.y, 
JTh Clare to or reflect a potential impact on safety o, misston/opera.tonal success. 


3.2 BASIC DEFINITIONS OF STATISTICAL TERMS 


Abscissa 
Alpha Error (a) 


Beta Error (0) 


Confidence Interval 


X coordinates in a rectangular coordinate system. 


The probability of rejecting a true population parameter or 
average. Also called Type I error or Producer s Risk (re- 
jected a good lot). 

The probability of accepting a false or incorrect population 
parameter or average. Also called a Type II error or Con- 
sumer’s Risk (accepted a bad lot). 

The interval computed around an estimated parameter, which 
expresses the probability of including the true population 
value within its bounds. 


Correlation 

Extrapolation 


Frequency Distribution 


The degree of relationship between two variables. 

A prediction of a Y value using an X value outside of the X 
range from which the model was derived. 

A tabular or graphical arrangement of data by classes, along 
with the corresponding class frequencies. 
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Independence 

Intercept 

Interpolation 
Least Squares 

Mean 

Normal Distribution 

Ordinate 

Parameters 

Pareto 

Pareto Diagram 

Population 

Projection 

Range 

Residuals 

Risks 

Sample Size 
Significance Level 


If the occurrence or nonoccurrence of one event does not 
affect the probability of occurrence of a second event, then 
these are independent events. 

The Y value when X = 0 for a line plotted in the X-Y coordi- 
nate system. 

Estimating a Y value between two known (X,Y) pairs. 

In cases where a linear relationship is known to exist, or may 
be reasonably assumed to exist, the principle of minimizing 
the square of the residuals is called least squares. Fitting a 
regression line through data points by minimizing the sum of 
the squares from the fitted line to the observed points. 

The term used to describe a population or sample_average. 
For a variable X, the mean is usually denoted by X or n. 

A type of symmetrical or bell-shaped frequency curve charac- 
terized by the fact that observations equidistant from the 
mean have the same frequency. 

Y coordinates in a rectangular coordinate system. 

The term applied to population or sample characteristics such 
as the mean and standard deviation. 

The concept that a relatively large percentage (80-90%) of 
problems will be caused by a relatively small percentage (10- 
20%) of related factors. 

A rank ordering of problem causes by their contribution, 
usually in decreasing order. 

Any group of items. A universe. 

An expansion of sample results to population values. 

The difference between the largest and smallest item in a set 
of data. 

Also known as random disturbance. The error in fitting a 
line/curve through a set of data points. 

The term applied to either or both the alpha or beta error. 

The number of items selected from a population that will be 
used to make inferences about the total population. 


Same as alpha error. Computed as 1 - (confidence level). 
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Slope 


The rate of change in Y per unit change in X for a line 
plotted in the X-Y coordinate system. 


Standard Deviation 

Universe 

Variability 


A measure of variability used in common statistical tests. It 
is the square root of the variance and is usually denoted by 
<j. 


Same as population. The total group of items possessing a 
certain characteristic(s). 

A term expressing the spread of items around a sample aver- 
age. 


Variance 


A measure of the spread or variability in a set of data. The 
average of the sum of squares of individual deviations from 
the mean and is usually denoted by a 2 . 
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ANALYTIC AND DESCRIPTIVE TREND TECHNIQUES; 
APPLICABILITY; EXAMPLES 


4.1 DESCRIPTIVE STATISTICS 

Descriptive statistics refers to raw data frequencies and tabulations, and simple measures such as the 
mean (average), median, and percentiles. While these quantifications are the most fundamental and 
elementary, they are probably the most important approaches in the NASA efforts for trend 
analysis. More sophisticated analyses should be preceded by the generation and examination of 
basic descriptive statistics. In many cases, a descriptive statistics approach, coupled with a graphical 
portrayal of the data, will be sufficient for trending purposes. 

4.1.1 Frequencies and Tabulations 

The frequency of a variable is simply a numerical count of the distinct values (or groups of values) 
for the variable. A standard application would be the number of problem reports from the PRACA 
Database on components of a subsystem (e.g., the STS ET) for a given time period; or the number 
of occurrences of each failure mode for a specific STS hardware component. In many cases, the 
first analysis step will be a frequency for the highest failing or highest problem-related criticality 1/ 
1R/1S components (usually the top 10-25). 

A 2-way tabulation is an n x k frequency table. Tabulations of this type are useful in investigating 
and portraying the frequency of occurrence for a pair of classifications or variables. In the trending 
effort, tabulation for the highest problem-related criticality 1/1R/IS parts for the current STS 
prelaunch period, and their respective counts for past periods, is useful to see if there is any 
developing trend. Such a tabulation would be of the form: 

Highest Problem-Related CRIT 1/1R Components for Current Prelaunch Period 

Subsystem = 




Launch Periods 

Component 

Criticality 

Current 

-1 

-2 

-3 








Note: Highest Problem-Related Components are determined for current period and historical fre- 

quencies are traced for previous periods to display possible trends. 
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4.2 GRAPHICAL TECHNIQUES 

The purpose for any collection of data is to convert data into information. This information is then 
used in support of management decisions. Graphical techniques provide simple, yet clear and 
concise means to transform raw data into information. A representative sample of the most 
commonly used techniques for quality assurance and reliability are presented in the following 
paragraphs. These techniques may be easily modified or combined to portray information. 

4.2.1 Scatterplots 

As emphasized throughout this Standard, one of the first steps in analyzing any set of data is to 
plot it. The first type of plot considered for trend data is a scatterplot. This plot is simply a 
bivariate plot with time on the x axis and the corresponding values on the y-axis. Additional 
information such as the regression line, moving averages or other smoothing procedures, and 
additional sets of y values, can be overlaid on this plot. 

Figure 4.2A is a sample scatterplot that shows the percent of welds failing inspection over a 16- 
week time period. 



WEEK 


Figure 4.2A. Scatterplot (Sample) 
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4.2.2 Control Charts 


Control charts are the basic grap f hl ^^ TJpIs o7 control charts, several 

way to present a large n 4 5 j Figure 4.2B is a sample control chart that is 

apphcabte for asan orbfter tile or wall thicKness of a heat exchanger. 



LOT 


Figure 4.2B. Control Chart (Sample) 


4.2.3 Bar Plots 

Bar plots are cotutnooly used .0 portray 

10 ,he graphka ' da,a ' 


Figures 4.2C through 4.2E are examples of bar plots. 
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Figure 4.2C. Bar Plot (Sample) 
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Figure 4. 2D. Segmented Bar Graph (Sample) 
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Figure 4.2E. Three-Dimensional Bar Plot (Sample) 


4.2.4 Pareto Charts 

Pareto charts place emphasis on the type of ^detect ,h™Pan?o Chart is a sorted 

graphical techniques, there are many ° n ( S _ ee figure 4 ? D ) and/or time period comparisons (see 
bar chart. However, by using segmented lb ( than through a s i mp i e bar graph. Note that 

figure 4.2G), much more information can be co y actual data rather than percent must 

when using pareto charts for a be/ore flM ^J r co arison (i. e ., if welds are 50% of 100 problems 
be plotted to provide an accurate pictu welds, a percentage pareto chart will 

l. - 5 » » .* 


OR'.G«*iAL P^=*E ,1> 
OF POOR QUALITY 
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TOTAL NUMBER OF PR* 


Figure 4.2G. Pareto Pyramid Chart 


4-6 




OR ‘G^.'AL 

Of; POOR 




Q ? f r g 

U^i ! I 


4.2.5 Histogram (Stem and Leaf) 

A histogram is a chart that displays frequency of occurrence. Such charts are essential when 
determining a probability distribution for the data. Histograms are developed by dividing 5 the range 
of ST data into equal intervals, counting the number of data pomts in each interval, and plotting 
these counts A stem and leaf plot provides more information than a histogram from the same 
data Instead of plotting points, the actual numerical data values are used to graphically represent 
Se^SSe. For example, suppose that we want to determine if the distribution of a particdar 
process is approximately normal. Figure 4.2H shows shaft diameters ranging from 1.016 to 1.070 . 



Figure 4.2H. Histogram (Sample) 


The same data shown in figure 4.2.H also can be displayed as a stem and leaf plot as shown below 


1.01 

1.02 

1.02 

1.03 

1.03 

1.04 

1.04 

1.05 

1.05 

1.06 
1.06 
1.07 


6 

011234 

6688899 

01122333444 

55667777788899 

0000011112233334444 

5556677777888999999 

01111112233333333344 

5555566667778888999 

0000011122233334 

56668899 

0 


The first line represents 1.016, the second line 1.020, 1.021, 1.021, 1.022, 1.023, 


and 1.024. 
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4.2.6 Smoothing 


Smoothing is a technique used to adjust time-series data for characteristic movements or variations. 
These techniques are usually classified into four main types: long-term or secular, cyclical, seasonal, 
and irregular or random. The methods that are used to smooth data include moving averages, 
moving medians, moving midpoints, and exponential smoothing. The exponential and moving 
average techniques are the most applicable and are developed in the next section. 

Figure 4.21 shows how smoothed data allows one to visually discern long-term trends and data 
patterns that are obscured by fluctuations in the raw data. 



Figure 4.21. Smoothed Data (Sample) 
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4.2.7 Box Plots 


s*s —<= 

a simple box plot without the x-axis is shown in figure 4.2J. 



MAXIMUM 


UPPER QUARTILE 


MEDIAN 


LOWER QUARTILE 


MINIMUM 


Figure 4.2J. Box Plot (Sample) 

times the interquartile distances from either quartile as asterisks. 
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4.2.8 Location Plots 


Location plots graphically represent the location and the type of error/defect occurring in a 
process. These charts are maintained over time; therefore, certain trends usually can be observed as 
they develop. For example, a circuit card may exhibit the pattern of solder defects, as shown in 
figure 4.2K. 



DEFECT 

MULTIPLE 

DEFECTS 


Figure 4.2K. Circuit Card Solder Defects 


The area surrounded by the dotted line is a candidate for a thorough investigation. Redesign may 
be necessary to enable this soldering to be achieved with higher quality. The chart also may be 
drawn using more descriptive and multiple indicators for defects, such as defect type, machine 
operator shift, and day of week. 

4.2.9 Examples 

4.2.9. 1 Example 1: Goddard Satellite Missions - 1986 Anomalies. There have been 49 Goddard 
satellite missions since 1970. The following Pareto charts aid in classifying the anomalies for 
satellites in orbit in 1986, and hence, the possible general failure modes. These satellites are: 

ERBS NOAA-6 

GDES-4 NOAA-9 

GDES-5 NOAA-10 

GDES-6 SMM 

LANDSAT-5 TDRS-1 

NIMBUS-7 

The Pareto chart approach is used to globally ascertain the problem areas and rank problems by 
frequency. Figures 4.2L through 4.20 identify satellite anomalies by: satellite subsystem; criticality 
of the failure; type of failure mode; and impact of the failure on satellite functioning. 
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NUMBER or ANOMALIES 


ORIGINAL PAGE IS 
OF POOR QUALITY 



Figure 4.2L. 1986 Anomalies by Subsystem 


30 


28 H 



MINOR NEGLIGIBLE MAJOR 

CATEGORY 


Figure 4.2M. 1986 Anomalies By Criticality (Mission Effect) 
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NUMBER OF ANOMALIES 


PAGE fS 

OF POOR QUALITY 



UNKNOWN OTHER (W/EXPLAN) PART PROB DESIGN PROB ENVIRON PROB 

CATEGORY 


Figure 4.2N. 1986 Anomalies By Failure Category 

30 -r 

28 - 
26 - 
24 - 
22 - 
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INDETERM STS OR INST DEGRAD NONE COMPONENT FAILED PART FAJUED SYS OR INST FAIL 

CATEGORY 


Figure 4.20. 1986 Anomalies By Impact 
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4 2 9 2 Example 2: Distribution of SSME UCRs/HPOT; FMEA Attributable UCRs. The follow- 
ing pie chart shows the distribution of Unsatisfactory Condition Reports (UCRs) written against 
shuttle main engines from January 1981 through December 1987. UCRs report both failures and 
other unsatisfactory conditions or potential problems; the reports follow SSME components from 
manufacturing acceptance tests throughout the component life-cycle. During the January 1981 
through December 1987 period, 6,002 UCRs were written against SSMEs, which encompassed 25 
shuttle launches, 764 engine tests, and 219,311 seconds of engine operation. 

Restricting attention to the High Pressure Oxidizer Turbopump (HPOT), 730 UCRs were written, 
217 of which could be directly related to FMEA/C1L failure modes. (Other UCRs referenced 
design/development, fabrication/quality, non-failure condition, etc.) The table summarizes the type 
of failure (referenced by FMEA code) on a quarterly basis. Turbine blade cracking/damage, bearing 
problems, and turbine disk failures accounted for 38.2%, 21.2%, and 17.5% of the FMEA applica- 
ble UCRs, respectively. No increasing or decreasing trends are evident with respect to these kinds ot 

HPOT problems. 
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UCRs 

Combustion Devices 

1370 

Fuel Turbopumps 

1173 

Oxidizer Turbopumps 

894 

Pneumatic Controls 

66 

Propellant Valves 

243 

Actuators 

205 

Controller 

680 

Harnesses 

159 

Igniters and Sensors 

462 

Lines, Ducts, Joints and Orifices 

305 

Fascos 

35 

Engine Systems 

213 

Non-Engine UCRs 

177 


Figure 4.2P. All SSME Components Proportion of UCRs (January 1981 - December 1987) 

Remark: In general, bar charts are preferable to pie charts. When using pie charts, the relative 
percentages (or absolute frequencies) of each segment should be given. 
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ORIGINAL PAGE IS 
OF POOR QUALITY 


Table 4,2-1 

SUMMARY OF FMEA MODES AND OTHER UCRs 
HPOT (B400) - DISTRIBUTION OF UCRs 


FMEA MODES UCRs/CRIT* 


1981 
2 3 4 


1982 


12 3 4 


1983 

12 3 4 


1984 

12 3 4 


1985 


2 3 4 


1986 

12 3 4 


1987 

12 3 4 


TOTAL 


02B Nozzle Vane Damage/ 1R 
03A Turbine Blade Cracks/ 1 
03C Honeycomb Retainer 
Failure/1 
03D Turbine Blade Impact 
Damage/ 1 
03F Loss of Damper Function/ 1 
04C Blade Shroud Chipping/ 1R 
05 A Interstage Seal Damage/ 1R 
06B Bending, Cracking of 
Turbine Vanes/ 1R 
07F Turbine Disk Failure/1 
08A Fracture, Distortion of 
Inlet Vane/1 R 
09B Impeller Blade Damage/ 1 R 
09C Balance Cavity Leakage/ 1 R 
13A Bearing Failure/1 
13E Loss of Bearing Nut 
Preload/ 1 

16A Excessive Primary 

Secondary Turbine Seal 
Leakage/ 1R 

17A Labyrinth Seal Leakage/3 
20A Fracture/Blockage of 
Coolant Circuits/1 
20B Coolant Passage Cracks/1 
21 A Weld, Parent Metal, or 
Drain Line Failure/1 
23A Turbine Piece Part 
Structural Failure/1 
25A Leakage Past the Inboard 
OPB/HPOTP Seal/3 


1 l 4 


14 2 1 


6 8 2 3 


2 2 2 3 
1 

2 115 

2 

2 l 3 


l 

1 l 


1 3 l 


1 1 
1 


2 3 
4 3 3 1 

1 


2 2 3 2 

I 


l 2 


2 3 


1 2 2 
1 

1 

1 2 2 
4 5 6 1 

1 


I 6 2 
I 


1 1 

1 1 1 


1 1 


13 12 


2 

1 

1 1 
l 


4 3 2 


l 1 


1 3 


6 

14 

1 


11 

54 

2 

1 

38 

6 

6 

1 

45 

1 


2 

2 

1 

7 

11 

4 


* “WORST CASE” CRITICALITY 
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4.3 SMOOTHING TIME-SERIES DATA TO PREDICT NEXT-EVENT FREQUENCY 


Frequencies or number-of-events from past time periods are often used as predictors of the fre- 
quency (or value) for the next time period. For example, the number of cannibalizations for 
previous shuttle flights should be reliable predictors for cannibalizations in the next flight, assuming 
conditions remain the same. 

Two methods are given for forecasting the next-event value based on past values. These methods 
are exponential smoothing and moving averages. With exponential smoothing, past event values are 
weighted by varying powers of a constant a, 0<a<l; more recent past events are given more 
weight than events occurring further back in time. The moving averages method has two purposes: 
(1) as a predictor of the next-event value, a simple average of the previous k events is computed 
and used as the forecast; and (2) time-series data can be smoothed so that an overall upward or 
downward trend becomes more apparent. Both methods are described in more detail in the follow- 
ing paragraphs. 

4.3.1 Exponential Smoothing 

Exponential smoothing is a method used to average past quantities to predict the value or quantity 
for the next (future) time period. The prediction or forecasted value for the next time period is 
simply a weighted average of the actual observed value for the current time period and the 
previously forecasted value for the current time period. The method is defined as follows: 

Let X n denote the actual value for the time period, n. By convention, the subscript n will denote 
the current time period, n-1 will denote the previous time period, and n+1 will denote the next 
time period. Then, 

F n + 1 = aX n + 0-“) F n (l) 

where 0<a<l, F n is the value forecasted for period n, and F n + , is the projected forecast for the 
next time period. 

Since F n is itself a weighted sum of the previous period actual value, and the previous period 
forecasted value, it follows that an alternative expression for the projected forecast, F n + 1 is: 

F n+1 = aX n + a (1-a) X n _, + a (1-a) 2 X n _ 2 + . . . (2) 

Thus, the next-period forecast gives a weight to each past observed value, with decreasing weights 
for observations in the more distant past. Observations in more recent time periods are weighted 
more heavily (i.e., have a greater contribution to generating a forecast). 

Remarks: (1) One problem in using exponential smoothing is to determine a suitable weighting 

value, a. Note first that a small value for a implies a slow response to change, and 
that a large value of a (i.e., a value close to 1) gives extreme weight to the most 
recent event(s). One way to choose a value for a is a retrospective simulation: 
consider all time periods before n; compute a forecasted value for the time period n 
for different values of a; then choose that value, a 0 , which gives a forecast (F n ) that 
is closest to the observed actual value X n ; use this value, a 0 , as the weighting 
coefficient for forecasting the next time period value, F n + 1 . 
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(2) Since the forecasted value will lie on the line segment between the actual value, X n , 
and the previously forecasted value, F n , the next forecast will lag behind any 
continuing trend. Using the trend slope, the previous forecast can be modified to 
offset the lag: letting b denote the slope of the trend line, then 

F n+1 = aX n + (1-a) [F n + b] (3) 

gives an exponential smoothing adjusted for trend. 

(3) There are several different conventions or variations in the calculation of the trend 

slope, b. Exponential smoothing adjusted for trend is a specific case of the 

ARIMA (autoregressive integrated moving average) technique. Standard texts on 
time-series analysis will provide more details on this method. 

4.3.2 Moving Averages Method 

Given past time-period measurements, the moving-averages method can be used to forecast the 
value for the next time-period. The forecasted value is simply the average of a fixed number of past 
values. For time-series data, the choice of the number of fixed values, k, to average is usually a 
year or a fixed, cyclical period. 

Let x., x„ . . ., x denote past time periods with measured/observed values y,, y 2 . • • • > 5V In this 

notation, y n is the" current or most recent time period. Using the last k measurements, the predicted 

value y n+1 for the next time period, x n + l , is given by: 

y.*, - 4- " W <4) 

j=o 

(i.e., a simple average over the last k time periods.) 

A more important use of a moving averages technique is to smooth data fluctuations, thereby 
accentuating a trend. Each data value is replaced by the average of itself and surrounding values. In 
this way, extreme fluctuations are minimized and an overall trend is displayed. 

Computationally, let k be the number of samples on either side of a data value that will be used in 
averaging J When, for example k = 2, then 5 values are averaged to give the averaged value at y^ 
denoted y- r In this case, 

- _ yj -2 + y»-i + Yi + y.+i + vi +2 (5) 




In general, the new averaged value is computed by: 


1 

yi = 2k + 1 ^ ^K‘- k ) + il 

ZK+ j = 0 
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Remark : Each y in Equations 5 and 6 is a centered-average. Problems will occur at the end points 
because there will not be enough values to extend out or back k-values for averaging 
purposes. These points can be dropped if there are a sufficient number of other data 
points in the sample. Otherwise, a modified average can be computed by repeating the 
end point values (i.e., extending them) up to k time-periods. 

4.3.3 Examples 


4.3.3. 1 Moving Average: Estimated Reliability of Vehicle Launch. An overall quantitative esti- 
mate of the reliability and the trends in reliability of vehicle launch for Atlas/Centaur, Scout, and 
Delta rockets can be made easily. The estimated reliability, in this context, means the crude ratio of 
successes to total launches (hence, the probability of success). 

Design changes and preflight testing procedures have occurred in the 1960-1988 time period. A 
simple ratio of total successes to total launches (for each vehicle type) would not reveal the 
reliability in later years, affected by such changes. Therefore, the data are presented by year. 
Figures 4. 3 A through 4.3C give the launch records for: Atlas/Centaur, Delta, and Scout rockets, 
respectively, in the 1960-1988 time frame. 



CALENDAR Y EAR 

[7*71 SUCCESS IX XI failure 


Figure 4.3A. Atlas/Centaur Launch Record 
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CALENDAR Y EAR 

(Z71 SUCCESS IX \l FAILURE 


Figure 4.3B. Delta Launch Record 



CALENDAR Y EAR 

[X7H success IX XI failure 

Figure 4.3C. Scout Launch Record 
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In order to minimize the fluctuation of reliability in a given year due to a small set of launches, a 
5-year moving average on yearly reliabilities is used. There are several ways to compute the average 
yearly reliability. One method would be to average the reliabilities around the given year. However, 
such a method gives equal weight to the reliability for a year with few launches and to the 
reliability for an adjacent year, which (by comparison) has many launches. To avoid this, a 
weighted moving average is computed. A weighted moving average is a modification of the moving 
average method described in section 4.3.2. The average value at y k , rather than being 


yk 


y k -2 + Yk-i + yk + Vk+i + Vk+2 
5 


(7) 


is computed as 


y k = 


_ W k-2 Y k-2 + W k-l Y k-l + W k Y k + W k+l Y k+l + W k+2 Y k+2 


EW: 


( 8 ) 


Here, W, are the weights, with the standard moving average having all weights equal to 1. Letting 

<*> 

J EW ; 


note that an equivalent expression for y k is 


y k = W k _ 2 Y k _ 2 + W k .,Y k ., + ... + W k + 2 Y k + 2 


( 10 ) 


where 


EWj 


(ID 


For the reliability of vehicle launch, the weighting system will be the number of launches attempted 
in that year. Then, reliabilities determined from a large number of launches will have significant 
weight and, conversely, reliability values from a small number of launches will have less significant 
weight. Figure 4. 3D gives the weighted moving average launch reliabilities for Atlas/Centaur, Scout, 

and Delta. 
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CALENDAR YEAR 

□ SCOUT + DELTA O ATLAS/CENTAUR 


Figure 4.3D. Launch Vehicle Reliability (Five Year Moving Average) 


As an example, for a specific computation, the moving average Scout reliability at t = 1 965 is 
computed as follows: the reliabilities for 1963 through 1967 are 5/9, 10/11, 5/5, 9/9, and 7/9, 
respectively. Since there was a total of 43 launches during this period, the weights are 9/43, 11/43, 
5/43, 9/43, and 9/43, respectively. Therefore, the computed moving average is given by: 

y 1965 = 9/43(5/9) + 11/43(10/11) + 5/43(5/5) + 9/43(9/9) + 9/43(7/9) 

= 36/43 = .837 
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4.4 FITTING TREND LINES - REGRESSION 

4.4.1 Overview and Distinction Between a Trend Line Fit and Regression 

4.4.1. 1 Overview. Intuitively, suppose at a set of points x t , x 2 , • - x n , w e are given a set of 
measurements/experimental values/observations y t , y 2 , • • •, y„- ° ne can display these data points 
( X j, y;> graphically as shown in figure 4.4A: 


Y 



We suspect that there is a relationship between the observation points {xj} and their corresponding 
observables {yj. The simplest case is when the relationship is linear, so that graphically we have, 
approximately, a line as shown in figure 4.4B: 



Figure 4.4B. Approximate Linear Relationship Between the Observation Points or Times {x,} and 

the Observed Values {y ( } 
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When the points {x,} are time values, this linear relationship, should it exist, is called a linear 
trend; and the process of finding the best line to fit the data is called fitting or determining a trend 
line. Fitting a trend line can be regarded as relatively simple, or somewhat sophisticated, depending 
upon the point of view. The main subtleties and complexities arise, not from determining the 
equation of the trend line, but rather in interpreting how accurately this line fits the data. In the 
remainder of this section both the simpler (more practical) viewpoint and a more in-depth presenta- 
tion are given. 

An overview of the process for fitting and evaluating the accuracy of a trend line proceeds as 
follows: 

a. Find the equation of the trend line. 

b. Determine whether the trend line, as the best fitting line, adequately models or fits the 
observed data. If there is a significant lack of fit, then there is no identifiable linear 
trend. 

c. Assuming that the trend line adequately fits the data, test to see if there is a positive or 

negative (i.e., upward or downward) trend. This requires a statistical test, using what is 
known as a t-distribution. Briefly, the reason one employs a statistical test is because 
there is rarely a perfect fit - that is, the {y^ values rarely lie directly on a line. In 
assuming that a true linear relationship holds between {xj and (yj, the deviations 
between each observed y i and its corresponding point on the trend line are, therefore, 
considered errors in the measurement of the y it or necessary variations in y for a given x ( . 
These errors, in turn, mean that the actual coefficients of the computed trend line have 
some error or variation in them. The question then becomes: can one rule out the 

possibility that the trend line is flat (i.e., no trend), taking into account the errors in 

estimating the coefficients of the trend line? 

d. Finally, assuming that there is a non-zero trend, use the trend line to predict future 
values and the confidence limits or probable errors surrounding these predicted values. 

4.4.1. 2 Distinction Between a Trend Line and Regression. Strictly speaking, fitting a trend line is 
slightly different from the statistical method known as regression analysis. Determining the equation 

of the best fitting line is the same in both cases; the methods differ in how one interprets how 

accurately this line fits the data, and how one A interprets the errors or deviations from the observed 
values {y ; } and the predicted, fitted values {yj. 

In fitting a trend line, the assumption is usually made that for a given Xj the corresponding yj is an 
exact value. For example, if one is trending the number of significant problems reported by months 
prior to launch, the number of significant problems is a constant value for a given month. No error 
in measurement is postulated. In turn, any deviations between the observed values and the trend 
line values is attributed solely to lack of fit of a linear model. 

The regression analysis viewpoint is more subtle. Note that for each x ; there is a measured or 
observed value y;. There could be errors in measuring the value y ; precisely. For example, in a 
performance trend, if y; is a continuous variable (e.g., temperature), there will be some variation in 
measurement. Also, it may be the case that for a given Xj there is not a fixed y i( but a range of y s 
values. 
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X there is a range of values for y. And that in fitting a line, called the regression line, one is 
fitting each x to the mean value of the corresponding observable y. Therefore, at a given x, there 
corresponds a distribution of values. The usual assumption is that each measured y s comes from a 
normal distribution; and the variance of each distribution, denoted o 2 , is the same (see figure 4.4C). 


Y 



EACH OBSERVED VALUE IS ASSUMED TO COME FROM A NORMAL DISTRIBUTION 
CENTERED VERTICALLY AT A GIVEN THE VARIANCE OF EACH NORMAL 
DISTRIBUTION IS ASSUMED TO BE THE SAME. 


Figure 4.4C. Regression Line 


The important point with the regression assumptions is that differences between the obser e <? 

(y i and the corresponding predicted values {y j} can be attributed to the necessary random 
(i e the distribution) of each yj. Assuming that a line fits the data, the distributional properties of 
y at each x, allow for quantitative statements about the confidence limits, or bands, in the estima e 
slope coefficient, and in values predicted by the fitted line. 

4.4.2 Determining a Trend Line by the Least Squares Method 

It is evident, from a geometric viewpoint, that there may be many candidates for the best fitting 
line For a given line that seems to best approximate the data, a slight perturbation of the line ca 
also be regarded as a good fit. Therefore, a precise notion of the best fitting line is needed. 

By definition, the line that best fits the data point { Xj , y*} is that one where the sum of the squares 
of the deviations between the line at each >4 and y, is a minimum. Figure 4.4D illustrates this 

concept. 
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Th« “faHt" Iki* to ysifbx ouch that Iho sum of ill •fltofid 
deviation* to • minimum. 


Figure 4.4D. Trend Line (Least Squares Method) 


To specify the least squares requirement algebraically, note first that the equation of a line is 
y = a + bx * 

where a is the y-intercept and b is the slope. 

Thus, the equation of the fitted line will be of form: 


A , u 

y = a + bx 

where y denotes the predicted or fitted values. 

Requiring that the sum of the square deviations is a minimum, means that 

e (yj-y i ) 2 

i 

is a minimum, where yj is the value of the fitted line at Xj (i.e., yj — a + bx^). 


(14) 


(15) 


Since 

E (y r y j ) 2 = E [ yi - (a + bXj )] 2 
i i 


(16) 


4-25 



the minimization requirement really means: find the values for a and b to make the sum of squares 
the smallest. By standard calculus methods (take the partial derivatives with respect to a and b, and 
set to zero), the solutions for the slope b and intercept a are: 


. n Ex^ - ExjEyj _ E(x r x)(y r y) 
b = — * 


n L \ { 2 - (EXj ) 2 


E(Xj-x ) 2 


(17) 


a = ^ x i 2 ~ = y - bx 

n EXj 2 - (Exj ) 2 

where x and y denote the means of the {xj and {yj; i.e., 
Exj 


x = 


n 


y = 

n 


(18) 


To summarize the above, the trend line is that line y = a + bx, which will minimize the sum of 
squared deviations between the trend line and the observed values {yj. There is a unique solution 
for this line, given by: 


y = a + bx (general form) 

b = E(x i -x)(y i -y) 

E(xj-x) 2 

a = y - bx 


(19) 


Section 4.4.8 gives an example demonstrating the actual calculation of a trend line. 

4 4 . 2.1 Indepth/Theoretical Point-of-View. (This section is not needed for the basics of obtaining 
a trend line, and consequently may be bypassed.) The focus is to present a more rigorous founda- 
tion for the least squares method (and hence, trend line fitting). Such ideas give a greater under- 
standing of the least squares method as an estimation method. In turn, generalizations beyond a 
linear model, and questions about the accuracy of fit, are more easily approached and understood. 

The least squares principle is an estimation method, for which the basic principle is: at the 

observational points x„ x 2 x n we have measured values ft. y 2 , • • •• V measured 

values may not be the exact true observable values jj,, ^ 2 , • • •* Vn- This cou * d be due t0 er I orS ' n 
measurements, or the fact that the true, unknown, y s are really mean values for a fixed x,. But we 
assume that a theoretical model exists, which gives the true {yj given {xj. This model is expressed 
as a functional dependence: 


y = f(9„ 0 2 . ■ • • . ©ki x i- x 2* • • • • x n) 
where the 0j are the parameters of the model. 


( 20 ) 
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By the least squares method, the best choices (i.e., estimates) for the unknown parameters {0,} are 
those that minimize the quantity: 


X 2 = E [y; - f(0j, 0 2 > • • • » ©ki x i> x 2> • • • ’ x n)l 2 


( 21 ) 


In the case of a line (as the model we want to fit), 


y = 0, + 0 2 x (22) 

Since we want the parameter values (0,, . . 0k) that minimize X 2 , view the 0, as variables and 

minimize by equating all partial derivatives to zero. In the case of fitting a line: 


y = 0, + 0 2 x 

then 


(23) 


Then 


X 2 = 


n 


E (Vi 
i= 1 



n 


E ( Vi - 0,-0 2 Xj) 2 

i = 1 


3X 2 

30 , 


n 

E 

i= 1 


(-2) (Yi - 0 r 0 2 x i) = 0 


3X 2 

3 ^ 


n 

E 

i = 1 


-2x, (y, - 0 r 0 2 Xi) = 0 


(24) 


(25) 


Solving for the parameters 0,, and 0 2 gives: 


A = Ex, 2 Ey, - EXjy, Ex, 
nExj 2 - (Ex,) 2 


(26) 


A = nEXjy, - Ex,Ey, 
nEx, 2 - (Ex,) 2 

Strictly speaking, note that and & 2 are functions solely of the observations (x it y t ). Thus, and 
0, are estimators of the true, but unknown, parameters 0, and 0 2 . This means that, for each set of 
{(x , y )} observations, from the estimators 0,, 0 2 one can compute an estimated value of each true 
parameter 0, and 0 2 . In general, the distribution of these estimates about the true parameter values 
have certain optimal properties called unbiasedness and minimum variance. 


The final point is that, with y =0, + 0 2 x, the equation is linear in the parameters. In general, when 
trying to estimate the coefficients (i.e., parameters) of a higher order polynomial, 
y =0, + 0 2 x + . . . + 0 k + l x k , the parameter dependence is also linear, even though the equation is 
not linear in x. It will follow that the same methods used in estimating the parameters for a line 


4-27 



apply, and exact solutions exist. Thus, fitting a parabola or higher order polynomial can be viewed 
as a linear regression or estimation, as opposed to a non-linear problem. 

In summary, the least squares method is a technique for estimating parameter values of a model. 
One assumes that there is a functional model of observational points {x ; } and unknown parameters 
that predicts the true values associated with, each Xj. By minimizing the sums of squared deviations 

(least squares principle), estimators (0, 0 k ) of the true parameters (0„..., 0 k ) are obtained. For 

a given set of measurements {(Xj, yj)}, using the estimators will give an estimated value of the 
parameter. These estimated values are then used in the function f(0|, 02,...,0 k; X|,X2,...,x n ) to 
obtain a predicted value y for a given set of observational points Xj,X 2 ,...,x n . 

4.4.3 Determining Whether the Estimated Trend Line Fits the Data - Use of R 2 and x 2 Goodness- 
of-Fit Test 

4.4.3. 1 Intuition. Having determined the equation of the trend line, the next issue involves the 
degree to which a line adequately describes the data. Put another way, how good a fit is the line? 
To clarify the question of fit, figure 4.4E gives two different sets of data; both sets, however, have 
the same least-squares-derived trend line. 



It is intuitively clear that the line in figure 4.4E(a) describes the data almost exactly. In turn, 
qualitatively, there is a high degree of fit. With reasonable confidence, the estimated trend line can 
be used to predict future values based on future time (i.e. , larger x) values. In contrast, the data 
points in figure 4.4E(b) move up and down around the trend line, with large deviations. Intuitively, 
the trend line does not adequately model the data, and there is little confidence in prediction of 
future values. 

4.4.3. 2 R 2 as a Measure of Fit. The preceding section and figure show that a quantitative 
measure is needed to measure the precision, or accuracy, of a fitted trend line. One such measure is 
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called the R-square value, denoted R 2 . This measure is also known as the coefficient of determina- 
tion. 

To motivate the derivation of R 2 , let y = a + bx denote the fitted line. An obvious measure of how 
well y matches the observed yj values is to look at the difference between the observed and 
predicted values: €i = y ; - y,. If there is a close fit, these differences, called residuals, are small. 
Since only the magnitude, not the sign of each tj is important, we consider t; 2 = (y 4 - y j) 2 ; and as a 
measure of total fit, sum up the squared residuals, Ze t 2 . 

Now, a small (large) Zt- 2 would indicate a good (poor) fit. The problem with this measure is that 
there is no relative scale. That is, if the y x are large values, and y fits well, then €j could still be 
large, in contrast to the case when the yj are small, and tj is still comparatively large relative to yj. 

The way to normalize and evaluate the magnitudes of the e; relative to their corresponding yj, is to 
divide Ef 2 by the quantity E(yj-y) 2 . This quantity is called the variation about the mean, and is the 
sum of squared deviations about the constant line y = y, where y denotes the mean of {yj}. 


Consider the identity: 

(y r y) = (y.-Vi) + <ft-y> (27) 

Squaring both sides, and summing over the n observations i=l, 2, . . ., n, we get: 

e (y r y ) 2 = s (yi-y i ) 2 + E (y ^) 2 (28) 

The cross-product term 2E(y r y)(y r y j) can be shown to vanish. 

The measure, R 2 , follows immediately from the above identity. Note that the variation about the 
mean E(yj-y) 2 , has been partitioned into two sums of squares — t^e sum of residuals squared 
2 (y._y ,) 2 and the variation of the regression line about the mean, Z(y j-y) 2 . 

By definition, 

r 2 = £ ( y.~y )2 ( 29 > 

E (y r y) 2 

It follows from equation (28) that when R 2 is close to 1, then Efj 2 = E(yj - y j) 2 is close to zero. Tl^is 
is the desired criterion, since then the deviations between the observed y t and the fitted values, y it 
are small; i.e., there is a good fit. In contrast, when R 2 is close to 0, most of the variation about 
the mean (a fixed quantity) is not explained by the fitted line, but to the fact that most y ; do not 
lie close to the fitted trend line. 
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To summarize: 


R 2 (the coefficient of determination) measures how accurately 
the trend line fits the data. 

r2 _ E (y i-y) 2 , 0< R 2 < 1 

E (yi-y) 2 

R 2 close to 1 means there is a better fit 


( 30 ) 


Remark: 


The quantity R 2 derives from the identity E (y r y) 2 = E (y r yj ) 2 + £ (y r y ) 2 by dividing 
both sides by E (yj-y) 2 - Thus, 


i = s (y »-y i> 2 + r 2 

e (y r y ) 2 


(31) 


An equivalent expression for R 2 is: 


R 2 = 1 - £ t>2 (32) 

E (y r y ) 2 

4.4.3.3 Chi-Square, x 2 . as a Measure of Fit. The R 2 value provides a quantitative means for 
assessing how well a line fits the data. When R 2 is close to 1, there is a high degree of fit; when R 2 
is close to 0, there is a significant lack of fit. This measure, however, provides only a general, 
rather than precise, measure of fit. A chi-square test, denoted x 2 > can be used for a more accurate 
assessment of fit. 


To use the chi-square test, assumptions are made that the N measurements {yj are independent 
and normally distributed about their true means (for each corresponding Xj). This is equivalent to 
the assumptions that the errors = y—y ; are each independent and normally distributed with mean 0 
and variance (see section 4.4.1 for a more indepth explanation of these assumptions). By 
definition 


X 


2 - 


n 

E 

i= 1 



(33) 


is a chi-square variable with n-2 degrees of freedom. A small value of x 2 will indicate a good fit. 

4.4.4 Testing for a Non-Zero Trend — t-Test on the Slope Coefficient 

Assume that a trend line has been fit to the data. Therefore, one has the best linear approximation 
or fit: 


A , u 

y = a + bx 

where a and b are constants that have been determined by the least squares method. 


( 34 ) 
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further to determine quantitatively whether there is a linear trend. 

With the above assumption the next question is: does the fitted trend Jin. ™^7»”Xe"xht 
he is there a statistically significant upward or downward movement exhtbtted over time. Thts 

question has both a simple and a more complex answer. 

In the less rigorous, simpler approach, note that the constant b *v=s the slope of Jim trend line 
of^ increase (decreaseMn the variable y is b. Graphically, thts is depicted m ftgure 4.4F. 


* 



Figure 4.4F. A Non-Zero Increasing Trend 


however, the line is flat (i.e., has 0 slope). 
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(a) NO TREND DUE TO LACK-OF-FIT (b) MODEL FITS; BUT NO TREND BECAUSE 

OF O-SLOPE TREND LINE 

Figure 4.4G. Two Examples of Zero-Trend Lines 


4.4.4. 1 More Rigorous Basis in Testing for Non-Zero Trend. In determining the trend line 
coefficients a and b, no assumptions were made concerning the distributional properties of the 
observations | y • } To determine how reliable these estimated coefficiencies are, assumptions are 
made about the probability distributions of each y if or equivalently about each error term e [ = y l - 
y . The aim is to determine certain bands or confidence limits around the estimated trend line slope 
b'- to judge whether one can conclude that there is a statistically significant non-zero trend. Put 
another way, is b, in all probability, different from zero? 

More rigorously, one needs to determine the standard deviation of the slope b, and confidence 
intervals. The standard assumption is that the observables, y if are each normally distributed about 
this mean y • and that the y ] are independent. These conditions are equivalent to requiring that the 
error terms (also called residuals) are normally distributed with mean 0. Thus, in the linear model. 

y . = a + bxj + cj for i = 1, 2, . . .,n (35) 

These assumptions are shown geometrically in figure 4.4C. 

Based on the distribution assumptions for the measured values y s (or equivalently the ef it can be 
shown that 90% confidence intervals for b are given by: 

b ± l ( n ‘ 2 » 95 > * S ( 36 > 

[E (x r i) 2 l ' /2 

where s is the estimated standard deviation of the residuals. The value t (n _ 2i 95) is the 95% point of 
a t-distribution with (n-2) degrees of freedom. 
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The estimated standard deviation of tj denoted s, is given by: 

. A 1 1/2 

s = S (y r y i > 2 

n-2 


( 37 ) 


and the quantity 

s < 38 > 

[E (x r x) 2 ]' /2 

is called the estimated standard deviation or standard error of b. Letting 7 denote 


[E (x r x) 2 ] /2 

we can say with 90% confidence that the true trend line slope, 0, lies in the interval 

, (40) 

[b - 7, b + 7] 

Note that if the above interval does not include 0, then there is a statistically significant trend (at 
the 90% confidence level). Hence, there is a non-zero trend. 

An equivalent way to test whether b is significantly different from zero (and hence, a non-zero 
trend) is as follows: 

\/ 2 

the variable t = bIE(X| ~ xH has a t-distribution. 


When there are n data points, there are (n-2) degrees of freedom. This number, n-2, characterizes 
the distribution completely. 

First compute t for the data sample in question. Next locate |t| (or the nearest value < |t|) in the 
statistical *table of t-values for n-2 degrees of freedom. (See Table 4.4-1 at the end of th, ^ ect J°^ 
The column that ltl appears in (labeled F in the table) gives the percentage (divided by 100) °' 
I^ribution that vvouM have this t-value or less - while having b equal to 0. This number is labeled 
ta b Therefore! a gives the probability to get the computed t-value or greater and still have no 
trend (Technically since we are considering |t|, the actual probability is a/2.) Thus, when a is 
small (sav 025) there is little chance of getting the computed t-value without a positive or negative 
trend Fo" a < 025 we conclude that the true value of the slope is. in fact, different from zero; 
hence there is a trend. Note that, in general, the higher the t-value, the greater the likelihood that 
the trend slope is different from zero. 

To further clarify the above procedure, suppose that there are 10 data points and the computed 
value of t is 2 15 Then, there are 8 degrees of freedom (i.e., n-2, when n = 10). This computed t- 
value is between the table t-values of 1.860 and 2.306. The percentage point of the distnb ration . for 
a t-value of 1.86 is .95. Therefore, there is only a 5% chance of getting a t-value of 1.86 or greate 
(and hence of 2.15) while having no trend. Since we are considering both t and -t, more techni- 
cally, the actual probability is .05/2=. 025 or 2.5%. 
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For practical purposes, whenever the probability of no-trend is 10% or less with the computed 
t-value, we will assume that there is a trend (i.e., b 0). Consequently, to test for a non-zero 
trend the following procedure can be used: 


(1) Compute t-value. 

(2) When there are n data points, there are n-2 degrees of free- 
dom. 

(3) Read down the .95 column of the t-distribution table to the row 
corresponding to n-2 degrees of freedom. 

(4) If the computed t-value is greater than or equal to the located 
table value, there is a significant non-zero trend. 


4.4.5 Prediction of Values Using the Trend Line 


The fitted trend line is given by: 


A 

y 


a + bx 


(41) 


However, both estimated coefficients,^ and b, are subject to error. This error, in tur^i, will 
influence the predicted value variable, y . If we ignore these errors, then the prediction^ y for a 
given value of x is straightforward. By simple substitution, when x equals x 0 , then y 0 = a + bx 0 . 
Usually x 0 represents a future time for prediction purposes. 


Simple substitution in the trend line equation give^ a predicted value. To ascribe confidence limits 
to this predicted value, the standar^ deviation of y 0 at a given x 0 is needed. It can be shown that 
the estimated standard deviate of y 0 is: 


estimated 


standard deviation 



}_ + (x 0 -x) 2 

n E (Xj-x ) 2 


Vz 


(42) 


where. 


k2 _ £ (yry i) 2 


(43) 


n-2 


A 


The 90% confidence limits for a predicted value y 0 arc: 


VO ± ^(n-2, .95) ' s 


1 + (xp-x) 2 


n E(Xj-x ) 2 


Vz 


(44) 


Here - t(n-2, .95) 
freedom. 


is the t-value corresponding to the 95th percentage point with n-2 degrees of 
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PERCENTAGE POINTS OF THE STUDENT’S t-DISTRIBUTION 

The .able gives values of fo, different degrees of freedom . such as .o produce specified values 
F(t a ;v), where 


„;«> -/ 


r['/2(u + i)] 


1 


F(t 


F(-t a ;v) = 1 - F(t a ;y), v = 


V™ rc/ju) , ~ t 2 i 1/2 ("+ 


dt = 1-a 
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The 90 % confidence limit means that there is a .9 probability that the true value of y at x 0 will lie 
in the interval 


r A A . 

[y o"y. y 0 + 7l 

where, 

(Xp-X ) 2 

E(x r x) 2 

To summarize: 


7 - ^(n-2, .95) ‘ s 


(45) 


(46) 


The trend line equation is 
y = a + bx, 

where y is the predicted value. 

The predicted value y 0 at a point x 0 is found by simple substitution. 

Confidence limits around the predicted value can be computed - see text and 
figure 4.4H. These limits or bands represent the most reasonable range for the true 
value of y at x 0 . 


i i r-r i 1 i i ""t 1 i i 
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Figure 4.4H. Fitted Trend Line and Extrapolation of Trend Line to 
Future Time Values (with 90% Confidence Limits on Predicted Values) 
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4 4 6 Extension of Linear (Line) Fit to Polynomial and Exponential Cases 

svs MSSKS K-JSIKS ri-ars.” - 


y = ^ + a t x + a 2 x 2 


y — ae bx 


(quadratic) 

(exponential) 


(47) 


Graphically, these models produce the curves shown in figure 4.41. 


v 





Figure 4.41. Quadratic and Exponential Models 


With regard to trending, a linear trend means that 
a constant amount for each unit of time. Hence, 


the variable of interest y increases (decreases) by 
the rate of change of y is constant: 


dy 

~dt 


k 


(48) 


In a quadratic trend, y increases (decreases) by an amount 
the rate of increase (decrease) is not constant ; but, the rate 


that is proportional to the time. Thus, 
of change of the rate itself is constant. 


’ *L- kt; ± 

dt dt 




(49) 
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With an exponential trend the amount of increase (decrease) in y per unit time is proportional to 
the magnitude of y: 


dy 

dt 



(50) 


Therefore, the percentage of change per unit time is constant: 
dy/dt = k 

y 


(51) 


Overall, then, the progression of trend models is: 


linear 

(increase/decrease 
is constant) 


-* quadratic 

(increase/decrease 
is linear) 


exponential 
(% increase/decrease 
is constant) 


4.4.6. 1 Polynomials of Order > 2. A polynomial of order n has an equation of form: 


f(x) = a 0 + a,x + a 2 x 2 + . . . + a n x n 


(52) 


where the a; are constants. 

Seldom, in a trending examination, is it useful to consider any polynomial of degree greater than 2. 
Polynomials of higher degree (n>3) usually have local maxima and minima: and, hence, are not 
very useful for discerning global upward or downward movements. Additionally, use of higher 
degree polynomials can be misleading, because given (n + 1) data points, a polynomial of degree n 
can be found that exactly fits the data (e.g., if there are 4 y-values corresponding to 4 distinct times 
x, through x 4 , then a cubic polynomial will exactly fit the data). 

For the above reasons, polynomial trending fitting should usually be limited to second-degree 
(quadratic) polynomials. However, the methods described below for quadratics are equally applica- 
ble for polynomials of greater degree. 

A quadratic polynomial has an equation of form: 

f(x) = a 0 + a,x + a 2 x 2 (53) 

The least squares method can be used to determine the best estimates for coefficients a 0 , a,, and a 2 . 
Strictly speaking the polnomial is linear in the parameters to be estimated (the a,); therefore, the 
estimation process is a linear regression (not a non-linear regression problem). 

Notationally, the fitted quadratic is written: 

y = a 0 + a,x + a 2 x 2 (54) 

where y is the predicted value for a given x, and the a ; are the estimated coefficients. 
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The least squares estimation method yields 3 equations in the 3 unknown parameters a^, a^ a 2 . 
These equations are solved simultaneously for a 0 , a ( , a 2 . The equations are: 


= a 0 n + a, EXj + a 2 Ex^ 

EyjXj = a 0 Exj + a, E Xj 2 + a 2 E Xj 3 
EyjXj 2 = a 0 Ex^ + aj Ex s 3 + a 2 Exj 4 


4 . 4 . 6 . 2 Exponential Trend Model. An exponential model is of form: 
f(x) = ae bx 

where a and b are constants. 


In a trending context, when b>0, the model will approximate a trend where the response variable y 
rapidly increases in relation to increasing values of x; (when b<0, the response variable y rapidly 
decreases in relation to increasing values of x). 


The above equation is non-linear in the parameter, b; however, the equation can be transformed 
into an equation that is linear in b and in In a; thus, the parameter £ (and a) can be estimated 
using the least squares method. Writing the predicted ox fitted value as y and taking the natural log 
yields: 


A 

y 


= ae bx 


In 


A 

y 


In a + bx 


(57) 


Regard In a as a new variable, a to give: 
In y = a + bx 


(58) 


Since the observed value {yj are to be fitted by {yj, it follows that In y, will bejnodeled by 
In y^a + bXj. The least squares method yields two equations in the two unknowns a and b: 


Eln yi = a-n + bE Xj (J9) 

Exjln yj = aEXj + bExj 2 

Solving these equations simultaneously gives exact solutions for a and b. The estimate for a is 
found from a ~ by transforming back: 


a = In a 
e a = a 


(60) 
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Having the least squares estimates, a and b, the best fitting exponential trend equation, y =ae b \ is 
then determined by simple substitution. 


4.4.7 Applicability 

The fitting of lines, polynomials of order >2, and exponentials are basic techniques in trend 
analysis. The methods described in the preceding sections apply to all types of trending: program- 
matic, performance, and problem. In particular, the methods discussed should be used for perfor- 
mance trending, since the variables measured are continuous and usually have an uncertainty in 
measurement. 


To determine which trending model, if any, is most applicable, the suggested first step is to plot the 
data. Inspection will often suggest the most reasonable analytic model. A quantifiable method to 
determine whether a linear, polynomial of degree 2, or exponential model is more appropriate is as 
follows: 


The more closely a model fits the observed data, the smaller will be the squared deviations 
between the predicted and observed values. Therefore, the smaller the quantity 


E (Yi-Mj) 2 
n - L 


(61) 


called the estimated variance about regression, the closer the fit. 


The constant L equals the number of unknown parameters in the model. The model with the 
smallest estimated variance about regression should be used. 

4.4.8 Examples 

4.4.8. 1 Programmatic Trends on Orbiter/Orbiter Tile Wad Closures. Figures 4.4J and 4.4K dis- 
play the number of Work Authorization Documents (WADs) at KSC in the weeks prior to shuttle 
launch. The tile WADS and the remaining orbiter directives are followed separately. There should 
be a declining number (hence decreasing trend) in open WADs as the time-to-launch approaches. A 
display of the magnitude of open WADs and the rate of decrease provide decision-makers with 
information on workloads, scheduling, projected overtime, manpower resource allocations, and 
general suitability of launch date. 
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□ TILE WADS + TREND LINE 


Figure 4.4J. Trend of Tile WAD Closure (as of 6/03) 
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Figure 4.4K. Trend of Orbiter WAD Closure (as of 6/03) 




448 2 Multiplexer/Demultiplexer (MDM): Problem Trends in Design, Manufacturing, and 
Electrical/Electronic/Electromechanical (EEE) Failures. Multiplexer/demultiplexers in the Space 
Shuttle Orbiter act as data acquisition, distribution, and signal conditioning units. The units accept 
digital information from their controlling units (lOP, PCM, or GSE) and convert or reformat the 
digitized information into analog, discrete, or series digital signals for shuttle subsystems. Con- 
versely, the MDMs can receive analog, discrete, or digital information from shuttle subsystems, 
digitize these signals, and transfer this information to their controllers. 

There have been 776 failures in the 1976 through 1986 timeframe. Table 4.4-2 summarizes these 
failures by manufacturing, design, and EEE in terms of where failures occurred. 


Table 4.4-2 

MDM FAILURES FOR 1976-1986 PERIOD 


Where Occurred 
(No. of Failures) 

Manufacturing 

Design 

EEE Parts 

Acceptance (442) 

53 

60 

329 

— 

Certification (12) 

0 

6 

6 

Field (318) 

8 

69 

241 

Flight (4) 

0 

2 

2 

TOTALS (776) 

61 

137 

578 


The following graphs (figures 4.4L through 4.40) depict the frequencies and trends in manufactur- 
ing, design, and EEE failures during the 1976 to 1986 timeframe. (The rate of MDM manufacturer 
acceptance and field tests are essentially constant; hence, no normalization is necessary.) 
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Remarks: General conclusions based on the above trend charts are; 

(1) MDM maturity has leveled off at about 50 failures per year, with the majority of 
failures being EEE. The EEE anomalies have been constant at about 40 per year. 

(2) There is no significant trend in manufacturing failures; but the low level of manu- 
facturing failures indicates a maturity in MDM manufacturing. 

(3) Design anomalies show a decreasing trend and are low in number. 

(4) Despite a large number of failures, only 4 failures have occurred in-flight. Accept- 
ance, certification and field testing have identified nearly all anomalies. 


4.4.8.3 Ga As Solar Array Power Supply - Hypothetical Performance Trend. In future long-term 
manned missions, regenerative H 2 /0 2 fuel cell systems may be used. The power producing cells will 
consume H-, and O z ; these reactants, in turn, will be created from electrolysis units (power consum- 
ing cells) using H,0, created as a product from the original fuel cell reaction. Solar arrays wil 
drive the water electrolysis units. (Currently, solar cell arrays serve as a power source on the 
TDRSS and can reach 36 w/kg.) 


Hypothetically, assume that a Gallium-Arsenic solar array functions as part of a regenerative power 
supply system. One kw of continuous power is required from the solar cell. Based on the orbit-sun 
plane orientation (hence, sunlight exposure), the solar array will produce a peak output of 2 kw, 
with excess power over 1 kw diverted to storage batteries. Particulate and optical radiation, among 
other factors, will decrease array lifetime. A 1.6 kw peak output (during an orbit) is deemed a 
lower bound for adequate power output; and the solar array has a projected lifetime of 2 years, 
with declining output from 2 to 1.6 kw. The objective is to determine if the declining array output 
is consistent with a 2-year lifespan (with 1.6 kw the minimum required peak output). 

In Figure 4.4P, peak output on a weekly basis was plotted over the initial 40 weeks of the mission. 
A linear regression (trend line) was fitted to the data and extended beyond the 40- week duration for 
predictive purposes. Inspection of the trend line shows that the minimal required peak output (1.6 
kw) will be reached in less than 104 weeks (approximately 80 weeks). 
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KILOWATTS OF OUTPUT POWER 





A more complete analysis is as follows: 


Basic Statistics 


No. of Observations 

= 40 

Degrees of Freedom 

= 38 

R 2 

= .977 

Slope (b) 

= -5.24 

Std. Error of Slope 

= .13 

Std. Error of Residuals 

= 9.61 


Equations Used 


(29) 

(17) 

(38) 

(37) 


Since the R 2 value is very close to 1, there is a significant fit of the trend line to the data. To 
obtain the value of peak kilowatt output after 104 weeks, one substitutes x = 104 in the equation. 

y = 2020.3 - 5.239x ^ 


to give 

y = 1475.4 


(63) 


Thus after 2 years of use, the peak output will be below the 1600 watts needed. Solving equation 
(62) for x (1600 = 2020.3 - 5.239x) gives the predicted time when 1600 watts will be available. 

Here x = 80 weeks. 

The question is: how accurate is the predicted value; i.e., v^hat are the confidence limits on y at 80 
weeks? The 90% confidence limits for a predicted value y 0 are given by: 


/A 


(Yq) * t (n-2, .95) 


1 + (X o~ X)2 

n" E (x r x) 2 


(64) 


where, 


s 


2 


£ (Yi-yi ) 2 

n-2 


(65) 


Here, t (n _ 2 95) is the t-value corresponding to the 95th 
freedom. 


percentage point with n-2 degrees of 


The predicted value at 104 weeks is y 104 = 1475.4 watts; and at 80, the predicted value is 1600 
watts (the minimum wattage required). We have: 


s = 9.61 
E (x r x) 2 = 5330 

l (n-2, .95) = 168 


(66) 
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( 67 ) 


Substituting these values into expression (64) yields: 


A 

y 104 

A 

yso 


39.7 

28.6 


Thus, from expression (67), the 90% confidence interval for the peak output after 104 weeks is 


[1435.7, 1515.2] 


( 68 ) 


Since this interval does not include 1600, it is not probable that the peak output will be at or above 
1600 watts after 2 years. 


At 80 weeks (the predicted lifetime using the fitted trend line), from expression (67) the 90% 
confidence interval is: 


[1572.6, 1629.8] 


(69) 


If that time, x 0 , is desired where the lower confidence limit is at or above A 1.6 kw with x o ^ >arge as 

possible, the confidence interval formula (64) can be used. Substituting y 0 + 2020 ^ , ^ 239 0 
K ormotinc tn 1600 and solving for x n will give this lower limit bounded below by 1600. 

'.“Sswtkt ThusV .0 75 leek, ,he,e is only a ,0% Ukelihood .ha, .he 

solar array will deteriorate below 1600 watts peak output. 


The Figures 4.4Q and 4.4R graphically depict these results. Both charts show the data, trend line 
and 90% upper and lower confidence limits. 



Figure 4.4Q. Solar Array Power Output (with 90% Confidence Intervals) 
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Figure 4.4R. 


Solar Array Power Output (with 90% Confidence Intervals) 


,,, % 
rturT^m 276 pstTom LPFt’ discharge) ,o appro, im a t e ly 6500 psia. Pressure boos, ,s 
ecessary to prevent cavitation in the main combustion chamber. 

he following examples use hypothetical 

nalyses are of two types: one tracks I HPFT ■ constant fuel How 
taximum HPFT rpms over a set of past flights. SSMb comro Trendmg rpm values for f ixe d 

ressures for the required thrust levels, a jus l g u (For a declining efficiency, due 

"^leS: nominal values to maintain required 

lischarge pressures.) 

Fable 4.4-3 gives HPFT nominal discharge pressure and rpm levels for three SSME-rated power 
evels: 

Table 4.4-3 


HPFT PRESSURE AND RPM LEVELS 


Power 

Levels 

HPFT 

Pressure-Discharge 

rpm 

65% 

100% 

104% 

3953 

6110 

6443 

27,157 

34,386 

35,361 
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Telemetry data can be used in several ways. This example uses data on a single flight to check for 
trends in HPFT rpms at fixed SSME power levels. As noted previously, increasing rpm levels to 
maintain constant discharge pressure can be indicative of pressure seal problems. 

Table 4 4-4 gives observed (hypothetical) rpm levels for the three SSMEs during ascent phase from 
1 1 minutes into launch to main engine cutoff. During this period the engines are throttled at 104% 
of rated power. It is assumed that HPFT discharge pressure was maintained at 6443 psia. 


Table 4.4-4 


SSME RPM LEVELS (HYPOTHETICAL) 


Elapsed Time 
(minutes) 

SSME, 

(rpm/10 3 ) 

SSME 2 

(rpm/10 3 ) 

SSMEj 

(rpm/10 3 ) 

1 1 

35.32 

35.37 

34.75 

1 5 

35.33 

35.41 

35.29 

1 9 

35.31 

35.47 

35.56 

2 3 

35.4 

35.51 

35.88 

2 7 

35.36 

35.59 

35.97 

3 l 

35.51 

36.12 

36.40 

3.5 

35.40 

36.08 

36.41 

3.9 

35.49 

36.22 

36.80 

4.3 

35.81 

36.24 

37.10 

4.7 

35.89 

36.37 

37.12 

5.1 

35.70 

36.31 

37.12 

5.5 

35.67 

36.30 

37.20 

5.9 

35.4 

36.19 

38.60 

6.3 

35.82 

36.22 

38.65 

6.7 

35.77 

36.25 

38.70 

7.1 

35.91 

36.17 

39.10 


By inspection it can be seen that SSME, and SSME 2 rpm levels at 104% power are constant and 
within the nominal value. It is clear that rpm levels for SSMEj are consistently above the nominal 
value. Fitting a trend line for SSMEj rpm levels yields the following statistics: 
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ELAPSED TIME (MINUTES) 


Figure 4.4S. HPFT RPM Levels (at 104% SSME Rated Power) 

The trend line slope coefficient is positive 

^ SnTrL'n° sys'tentatic trend, indicattng a problem with maintaining the re- 

quired turbopump discharge fuel pressure. 
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4.5 SMALL SAMPLE METHODS 

a _ nl . si7( . mav be small Consequently, the uncertainty for any inference 
["JZl wiU be* large (or equfvalemly. .he accuracy or confidence in .he es,ima.e is low). In cases 
where the data is sparse, non-parametnc methods are often used. 

.V, A tn reels that do not depend on assumptions about the underlying 

Non-parametnc methods refe .* f omes These tests apply to small samples because no popula- 
distnbution from which the samp • xhe methods presented are quality control 

remain .he mos. im P or.a„. cri.eria 

in the data analysis. 

4.5.1 Acceptable Level-of-Performance Charts 

One of .he Key ideas in analysis is .. ”impU 

should be developed during equipment d g • haft . a CO mplex example might be the 

SSME a reveal ^trends ST32 

Sfy.Also! aVgrestn mean, yet every point is out of specifica- 

tion, as shown in the following chart: 


PARAMETER 



Upper Spec. 
Mean 

Lower Spec. 
Time 


The mean is well within specification, but the process is not! 

for relatively small sample sizes with minimal loss of effectiveness. 

Most control charts are developed using 

includes 99.73% of the P°P u ' atl( ™ a * j s out of control, when in fact it is not (producer’s 

giving an erroneous error signal tha P . | imited (by today’s standards) computa- 

S v '“ *° have b “" 

an effective trade-off between producer’s (a error) and consumers ((3 error) risk. 
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Table 4.5-1 is an example of the fraction defective (np) chart. It is applicable to virtually any 
quality control process. (Refer to figure 4.5A for a graphical representation). 

Table 4.5-1 

DEFECTS IN SAMPLE LOTS 


Sample Lot Number Sample Size Number of Defects (np) 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


30 

30 

30 

30 

30 

30 

30 

30 

30 

30 

300 



11 

np= __= 3.3 (average number of defectives) 
10 

11 

p= _= .11 (average fraction defective) 

300 


a- V3.3(l-.l 1) = V2.937 = 1.71 (standard deviation) 

UCL = np + 3a = 3.3 + 3(1.71) = 8.43 

LCL = np -3 a = 3.3 - 3(1.71) = -1.83 (less than zero and therefore is set to zero). 
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UCl = 8.43 



5 6 7 8 9 10 


Figure 4.5A. Graphical Representation of the np Chart 


Since all of the data points are within the control limits, the process is “in-control.” It is extremely 
important to note that in-control does not necessarily indicate within-specification. A machining 
process could be producing a product as indicated by the np chart in the example (in control) where 
the process specification is 2.5 ±1%. The product is in control, but not within specification. 

In summary, control charts can be designed to fit almost any quality control (or performance 
trending) application. With the on-line, computer-based power available today, real-time control 
chart type analyses can be of enormous benefit. 

4.5.2 Run Test to See if Observations are Free of Systematic Trends 

A Run Test can be used to test whether two samples, {x n } and {y m } come from the same 
population. The basic idea is to combine or mix the two samples, and then order them by 
magnitude. On the assumption that both samples come from the same parent population, the 
ordered sequence of x’s and y’s should be reasonably mixed. It should not be the case, for 
example, that most of the x x appear at the lower end (in order of magnitude) and the y x appear at 
the upper end. 

By definition, a run is a sequence of the same symbol or type. For example, in the sequence x 3 x t 
y 4 y ( x 2 y 2 y 3 there are 4 runs: a run of 2 x’s, a run of 2 y’s, a run of 1 x, and a run of 2 y’s. 
When there is a reasonable mix, the number of runs for the (n + m) observations will be large. 
When the number of runs is small, it is more probable that samples {x n } and {y m } do not come 
from the same population. 

The run test technique can be modified for data obtained sequentially over time to see if the data is 
free from systematic trends. This is the modification that will apply to trending questions. The basic 
assumption is that all the observations, obtained at different points in time, measure the same 
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constant quantity, c 0 . In performance trending, c 0 is the desired mean performance value, 
general, c n can be any value, but usually is taken to be the median of the observations {x }. To test 
whether the values have a constant zero-trend at a specified value, c 0 is taken to be that value. 


Now if all the observations ostensibly measure the quantity c 0 , then the observations should hover 
close’ to c n , with equal probability of an observation being above or below c 0 . Let A denote an 
observation above c 0 ; and let B denote an observation below c 0 . Since the observations {x n } are 
ordered by time, a sequence of As and Bs results. If there is no trend of observations consistently 
above or below the constant c 0 , then the number of runs will be high. However, if the measured 
values are consistently above (or below) c 0 , or consistently below for awhile and then above for 
awhile, the number of runs will be small. Thus, a small number of runs would indicate that there 

are systematic trends. 


The number of runs, denoted r, has a distribution based on the sample size n and the assumption 
that the set {x n } consistently measures c 0 . This distribution has a mean equal to (n + 1) and variance 




n-1 


(70) 


The following statistical table gives the critical r-value (denoted r 05 ) at the .05 confidence level. 
This means that when the number of runs is less than or equal to r 05 , there is only a .05 
probability that the set {x n } consistently measures the constant, c 0 . In the trending context, there is 
only a 5% chance that there is not a systematic trend. Therefore, whenever the computed number 
of runs, r, is such that r<r 05 , we conclude that there is a systematic trend. 


n/2 

r .05 

Remark: The above run test may be viewed as a quantitative measure to accompany the visual 

inspection of a control chart. 


Critical Values for the Run Statistic* 
a = .05 




1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

- 

- 

- 

2 

3 

3 

4 

5 

6 

6 

7 

8 

9 

10 

11 


♦Comparing against a constant trend value, c 0 


4.5.3 Kolmogorov-Smirnov Test for Trends 

The Kolmogorov-Smirnov test is a goodness-of-fit test used to measure whether a given sample {x n } 
could come from a certain theoretical model. First the basic ideas in the test will be noted; then 
modifications that place the test in the context of trend analysis will be given. 
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A sample of size n, {x n }, can be reordered according to magnitude. Assume that the Xj have been 
so ordered and thus, Xj<x 2 ^ ... <x n . A cumulative distribution function (cdf) S n (x) is then 
defined by: 


S„(x) 


[ 0 

if x<x t 

1 1 

if X:<X<X: 

1 n 

1 1 

l 1 

if x^x n 


(71) 


The function S n (x) is thus a step function that increases by ^ at each of the observational points. 


The idea of the test is to examine the differences between S n (x), the cumulative distribution for the 
sample, and any theoretical cumulative distribution function, F(x). If the sample {x n } comes from a 
population with cumulative function F(x), then one would expect that the difference 


D n = max { | S n (x) - F(x) | } 


(72) 


over all points is not too large. 


Under very general conditions, D n has a distribution that is independent of any theoretical cumula- 
tive distribution. Moreover, the distribution of D n is sensitive for small samples (n usually > 10). 
This allows testing to determine if the sample could come from a certain distribution or whether 
that hypothesis can be ruled out. 


The Kolmogorov-Smirnov test can be adapted to test for a non-zero trend (or for any specific 
trend). To illustrate, let {y , , y 2 , . . ., y n } be the observed values corresponding to times {x,, x 2 , . . 
., x n }. The {y ; } corresponds to the ordered set {Xj} and, in this manner, is already ordered; each y s 
is viewed as a frequency at the corresponding time X;. Therefore, there is a cumulative distribution 
function, S n (Xj). S n is defined as follows: 


S Q = 0 x<x, 

S, = _l!_ x, <x<x 2 

E* 

S 2 = + h x 2 <x<x 3 


s _ y,+y 2 +-+y„-i 

3 n-l “ 

E* 

S n = 1 X n <x 


(73) 


NOTE: The cdf for the sample of n points is simply the cumulative sum of the frequencies, { y s } , 

normalized by the total frequency, Ey ( . 

We want to compare the cdf S n (Xj) with the theoretical cumulative distribution function determined 
from a constant set of frequencies. The comparison is illustrated in figure 4.5B. 
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X 1 Xj X3 X( Xj 


Figure 4.5B. Plot of Observed Values Against a Constant Frequency Distribution 

The theoretical cdf F(x) from the uniform distribution of c 0 at each point x ; is representative of a 
zero-trend, because the values are constant over time. Assuming that the x ; are equally spaced, then 
F(x) may be defined at the observation points {x,} as: 

F(xj) = 1 
n 

F(x 2 ) = l 

^ /n a \ 


F(Xj) = 1 


F(x n ) = 1 

Assuming the times are equally spaced for any constant function at the x^ the cdf for a zero-trend 
is independent of the specific constant. The value of F(x) at Xj is simply i/n, where n is the total 
number in the sample. 
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To use the Kolmogorov-Smirnov test in the trending context, the test statistic is 

D n = max { | S n (x) - F(x) | } (75) 

where S n and F(x) are as defined previously. The critical values of D n for various sample sizes, 
n, appear in Table 4.5-2 at the end of this section. In principle, the differences |S n (x) - F(x)| 
are examined for all points x. Since S n (x) is a step function, only the differences at the jump 
points, Xj, need be computed. Thus, for a sample of size n, 2n differences need to be 
examined and the maximum taken. The 2n values are { | S„(Xj) - F(Xj)| , |S n (Xj-l) - F(Xj)|} over 
the observation points {xj. 

For example, suppose the sample size is 5 and a 90% confidence level is adequate. The K-S value, 
d 05 , is .5095. Therefore, at the 10% significance level, if the largest deviation between S 5 (Xj) and 
FfXj) exceeds .5095, then the assumption of a zero trend can be rejected. 

As a final point, the table of K-S values shows that for small sample sizes, rather large differences 
between the cumulative distribution functions must exist for one to conclude that the data must 
deviate from a constant zero-trend hypothesis. 

To summarize the Kolmogorov-Smirnov Test: 


Test statistic is D n = max { j S n (x) - F(x) | } , 
where 

S n (x) is cdf of observed data and 

F(x) is cdf of uniform (constant) distribution. 

S n (x) = y » + + • ’ F(x) = 1 

Eyj n 

{for x i <x<x i + 1 } 

A large difference, D n , indicates that the observed 
data does not come from a uniform or zero-trend 
distribution. 


(76) 
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Table 4.5-2 

PERCENTAGE OF POINTS OF THE KOLMOGOROV-SMIRNOV STATISTIC 

The table gives values d a for specified values of a such that 
P(D n s d„) = 1 - a 

where the Kolmogorov-Smirnov test statistic, D n , for the sample of size n is the largest deviation 
between the observed cumulative distribution and the theoretical cumulative distribution. 
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4.5.4 Examples 

4.5.4. 1 Example 1: Elevon Servo Actuator - Performance Trend. The elevon servo actuator is 
used in the shuttle hydraulic system to control orbiter aerosurfaces (elevons). Current applied to the 
servo linearly increases the secondary delta pressure for hydraulic elevon displacement. The nominal 
servo breakaway pressure is 50 psia. 

Figure 4.5C displays secondary delta pressure as a function of input amperage (in millivolts). The 
normal operating curve is linear. Post-flight tests on actuator performance shows a decline in 
secondary delta pressure for a fixed input current level. In general, there is a decreasing trend in 
performance. (The trend techniques covered in this Standard do not include methods for comparing 
several curves. A sample of pressures at a fixed current should be used to quantify a trend). 



SERVO CURRENT 

NORMAL TREND + FLT 1 O FLT 2 


Figure 4.5C. Elevon Servo Actuator (Single Channel Ramp Test) 


Figure 4.5D displays simulated data on servo actuator breakaway pressures. Prior to flight, the 
breakaway pressure in a given actuator is measured. At a breakaway pressure of >450 psi, silting 
or other problems are occurring; a breakaway pressure of >850 psi requires removal of the unit. In 
the following control chart, tests 1 through 3 indicate a decreasing level of performance (increasing 
trend in breakaway pressure). Once the unit reached >450 psi breakaway pressure (test 4), the unit 
was desilted. Performance is monitored again for subsequent flights. The increasing trend in break- 
away pressure is evident. After seven flights, the unit should be replaced. 
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Figure 4.5D. Performance Trend Monitoring (Elevon Servo Actuator) 


4.S.4.2 Example 2: Transducer Calibration/Beam Momentum Constancy. The run test can be 
used to test for systematic deviations or trends from a constant value. This example presents actual 
data for calibration checks on a shuttle air data transducer assembly (ADTA) probe unit. It also 
presents a hypothetical example on time variation of an ostensibly constant beam momentum used 
for a high-energy bubble chamber experiment. 

a. ADTA - Calibration Trending. 

The air data transducer assembly is part of the guidance, navigation, and control system. Among 
other functions, the air probe unit is used in determining the angle and acceleration at reentry. 
Calibration checks on an ADTA-1 were made throughout the 1982 - 1984 period. Figure 4.5E plots 
the observed alpha pressures and a fitted trend line to the data. 


4-61 




0.02 


o.o) H 



-0.03 H 


- 0.04 -f — i — t — t — I — i — i — i i i i r i \ i i i i i i i r t i i i i i i r~i i i i i I 
123067 8 91011121 23456789 10 11121 23456789101112 

TIME (MONTHS) 
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performance range. 

Figure 4.5E. Air Data Transducer Assembly Calibration Check (ADTA-1) 


It is evident that there is a declining trend in calibration from the baseline of 0.0 inches of mercury 
(in. Hg.). The fitted trend line will give a numerical measure of the decline in calibration (slope = 
-.00083, standard error of slope = .000076, R 2 = .952). In addition, a run test can be used to give 
the probability for having the systematic trend of all sampled values below the baseline while still 
having a calibration distribution that randomly fluctuates above and below the baseline. 

The measured O-baseline calibration values are: 


Time 

Alpha Pressure (in. Hg.) 

Month 

Year 


9 

1982 

-0.002 

10 

1983 

-0.003 

2 

1984 

-0.003 

5 

1984 

-0.005 

7 

1984 

-0.008 

10 

1984 

-0.01 

3 

1985 

-0.015 

10 

1985 

i 

-0.023 
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Since all observed points lie below the prescribed calibration baseline of 0.0 in. Hg., the series 
yields a run sequence (with 1 run): 

BBBBBBBB 

where B denotes below the line y = 0.0. Using the run statistic table in section 4.5.3, the critical 
value at the significance level a = .05 and n/2 = 4 is r 05 = 2. (Note that there are eight points, with 
equal probability of being above or below the constant line; hence using previous notation, for 
{x '} and {y ' } as samples, n' = m'= 4.) Therefore, the probability of having only one run and 
no" systematic" 1 trend is less than .05. Hence, there is a statistically significant declining trend. 

However, there is always transducer error, so slight deviations from the 0.0 in. Hg. baseline can be 
considered errors in measurement rather than true calibration error. Although all measured values 
are below 0.0, they may fluctuate about the median and cancel out any systematic trends. The run 
test then can be used to check for systematic trends from the median. This approach also minimizes 
the error measurement, since the baseline is taken to be the observed median. 

The sample median is -.0065. A run sequence about the median is given by: 

AAAABBBB 

where A denotes above the median and B denotes below. The sequence has a high degree of order, 
with only two runs among eight symbols. From the run statistic table, the critical value for n/2 - 
is r ns = 2. Hence, the probability of having as few as two runs with no systematic trend is smaller 
than .05. The conclusion, already seen intuitively, is that the deviations in the ADTA calibration 
are not due to random errors, but represent a systematic declining trend from the prescribed 
calibration value. 

b. Beam Momentum Constancy 

The following example illustrates how, in small samples, the run test can be used to quantify non- 
random deviations or trends from a prescribed constant level of functioning. No significant linear 
trend need exist. Practical applications would include instrument calibrations (as above), sets ot 
tests for leaks prior to flight (i.e., constant pressure to be maintained), and alignment accuracies 
(e.g., HST fixed head stargazer). 

The following data are based on measurements of beam momentum in a bubble chamber experi- 
ment. The average momentum of the incident tracks on 20 rolls of film, ordered according to the 
time of exposure, are: 




GeV/C Units 





[rolls 1-7] 

18.61 

18.59 

18.72 

18.69 

18.75 

19.05 

19.06 

[rolls 8-14] 

19.03 

19.04 

19.13 

19.16 

19.19 

19.15 

19.07 

[rolls 15-20] 

19.08 

19.01 

18.91 

18.81 

18.90 

18.89 
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Figure 4.5F. Time Variation of Beam Momentum 


Figure 4.5F illustrates that the momentum appears to increase and then decrease during the expo- 
sures. A run test is used to determine analytically whether the hypothesis of constant momentum 
can be rejected. The median value for the observations is 19.02 [(19.03 + 19.01)/2 due to even 
number of data points]. The classification of values in relation to the mean, where A means above 
and B means below, yields the series: 

BBBBBAAAAAAAAAAABBBBB 

This series has 3 runs among the 20 symbols. The table for critical values of the run statistic (here 
n/2= 10) shows that less than 6 runs would occur only 5% of the time with a constant momentum 
beam and measurements deviating randomly about the median. Therefore, the hypothesis that the 
momentum remains constant is rejected. 
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Chapter 5 

SELECTION OF PARAMETERS (VARIABLES) FOR PERFORMANCE TRENDING 


5.1 OVERVIEW/BASIC APPROACH 

In performance trending, the values of variables (or synonomously, in this section, parameters) that 
either directly or indirectly indicate system/component performance and reliability are examined for 
trends. Therefore, variables that are measurable and related to component performance need to be 
determined. In analyzing a criticality 1, 1R, or IS component, it may be the case that no suitable 
physically measurable variables exist. However, in many cases such parameters and the capability to 
measure the parameter values over time does exist. The subsequent sections present several methods 
and criteria for identifying and selecting candidate parameters for performance trending. 

The selection of the parameters to be measured and trended is based, in large part, on engineering 
judgement. There are, however, statistical methods which can be used to select the most highly 
correlated parameters when suitable raw data on performance is available. A simple correlation 
coefficient can be computed to see if the parameter and component risk or performance are linearly 
related. Contingency tables can be used to see if there is any degree of association between risk/ 
performance and the potential parameter. And in the case where there are several candidate 
parameters, and one wants to select those that are the most significant in predicting risk/ 
performance, a stepwise regression technique can be used. Each of these methods is covered in 
subsequent subsections. 

As an overview, determinations for identifying performance parameters should be based on. compo- 
nent criticality; availability of sensor/performance data; trendability of sensor/performance data, 
and failure/problem history. The FMEA/CIL documents will provide criticality classifications. 
These documents, together with system assurance and assessment documents, should be examined 
for failure mechanisms and indicators that can be detected by flight monitoring instrumentation 
and/or ground checkout and testing. The trendability of data is related to: (1) the establishment of 
redlines or norms of performance; (2) the accuracy and timeliness of sensor data; and (3) the degree 
of correlation of the possible parameter with actual component performance. 

5.2 SIMPLE CORRELATION 

In the case where a performance characteristic or the risk of failure of a component is linearly 
related to another more easily measured variable, a simple correlation test can be used to quantify 
the degree of linear association. If data is available for correlation, and if a test of association or 
correlation is high, then the more easily measured variable is a reasonable candidate as a perfor- 
mance trending parameter. This section presents the computational aspects of testing for a simple 
linear correlation. 
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( 77 ) 


By definition, the sample correlation coefficient r, for a sample of size n, 

{(x,y,), (x 2 ,y 2 ), ••• .(x n ,y n )} 


is given by: 


n 

E (x r x) (y r y) 
i= 1 


’ n 

l /2 

n 

E (x r x) 2 


. E (y r y) 2 

i=l J 


i=l 


(78) 


The sample correlation coefficient measures the strength of a linear relationship between the ran- 
dom variables X and Y, even when the data does not come from a bivariate normal distribution. 
The values of r range between -1 and 1, with r close to 0 implying no correlation and |r| close to 1 
implying a high linear correlation. 

Since r is based on a sample , it is customary to perform a test of significance on the computed 
sample correlation coefficient. This will give the probability of obtaining the r value (or higher) and 
having no linear relationship between the random variables. In order to perform such a test of 
significance, the assumption is made that the samples come from a bivariate normal population. 

The significance test is based on the fact that the variable Z, defined by 



2 ' 1-r > 


is approximately normally distributed (with mean 0 and variance 1). 

A table of the standard normal distribution can then be used to test whether the r value obtained 
implies that a linear relationship exists. 

Suppose, for example, a sample of size 12 is taken, and a correlation coefficient of r = .7 is 
computed. 

Then 



(80) 


= 2.60 

Using a table of the standard normal distribution, there is less than a 1% chance of obtaining this 
z-value and thus this sample correlation coefficient, while having no correlation. Hence, the sample 
shows a high degree of correlation. 
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5.3 STEPWISE LINEAR REGRESSION 

The stepwise linear regression method can be used when there are several candidate variables for 
performance trending and where the aim is to choose one or a subset of these variables. More 
specifically, the aim is to isolate a subset of variables that will yield an optimal prediction equation 
with as few variables as possible. In performance trending the dependent variable y usually repre- 
sents levels of risk of failure (however, y can be any output parameter). The prediction equation is 
of the form 

y = a 0 + a,x, + a 2 x 2 +...+ a n x n . (81) 

Many statistical packages (e.g., SAS, SPSS) will perform a stepwise regression and sources in the 
References section describe the computational method in detail. The key point in the method, 
however, is as follows: 


The aim in any regression is to find the coefficient estimates a Q , a ]v ...,a n that will minimize the 
sum of squared differences between the observed value, y, and the predicted value, y (over the 
values of the variables {x;}). As noted in section 4.4.3 for the single variable case, the total 
variation or sums of squares can be partitioned into two sums: 

(y-y ) 2 = £(y-y ) 2 + E(y-y ) 2 (82) 

This identity leads to the definition of R 2 as a measure of fit, where R 2 ranges between 0 and 
1, and where R 2 measures the amount of total variation that is explained by the regression fit. 
In a stepwise regression (forward method) the variables are included one-by-one into the 
predictive equation in the order of highest contribution to explaining the total variance (i.e., 
those that make R 2 close to 1). In this way, the most significant predictor variables can be 
isolated. 

5.4 TWO-WAY CONTINGENCY TABLES - TESTING FOR INDEPENDENCE BETWEEN 
TWO CLASSIFICATIONS 

A two-way contingency table can be used to measure the degree of association between two 
variables or classifications. The relevance to parameter selection for performance trending is: (1) the 
risk of failure can be tested for association with a measurable performance variable; (2) two 
possible performance parameters can be tested against each other to determine if, in fact, they are 
independent; if they are highly dependent, only one needs be considered for performance trending; 
and (3) if a variable is known to be a reliable indicator of risk or performance, but is not easily 
measurable, another variable may be tested for its degree of association with the original perfor- 
mance variable; a high degree of association (dependence) suggests that the more easily measured 
variable may be used. 

The following discussion presents the basic concepts used in a contingency table analysis. For each 
observation, one has values for two different variables (e.g. temperature and pressure). The aim is 
to determine if the values are statistically independent of each other, or whether the value of one 
variable is associated with the value of the other. A matrix (contingency table) of the values is set 
up and a chi-square test is used to measure whether any association between variables exists. 

In general terms, suppose that two variables or attributes, A and B, are classified into I categories 
for A, (A,, A 2 ,..„A,) and J categories for B, (B„ B 2 Bj). Variable A categories could be risk 
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levels (or failure frequencies) for a component and va: table B categories could be measured down- 
line pressure levels. 


The counts for observations with combined characteristics Aj and Bj are denoted n^ and can be 

written as a contingency table: 


Contingency Table for Two-Way Classification 



Br 

b 2 


Bj 

n i 

A 1 

n n 

n I2 

n )3 

n u 

n,. 

A 2 

n 21 

n 22 

n 23 

n 2J 

n 2 . 

A 3 

n 3l 

n 32 

n 33 

n 3J 

n 3 . 

A I 

n ll 

n I2 

n I3 

n U 

n, 

n ' J 

n i 

n 2 

n 3 

n.j 

n 


The row and column marginal totals are denoted n ; . and n.j, respectively. 

The cell probabilities are estimated by = n v /n. If the cell counts are independently distributed 
(i.e., there is no association between variables A and B), then the probability for having properties 
Aj and Bj simultaneously is the product of the probabilities for separate occurrences. Thus, = 
P|. • Pj. and the proportion in row i and column j, n^n, should be 


n n n 

Put another way, if the probability of attribute Bj is independent of any attribute Aj, then the 
conditional probability P(Bj| Aj) is the same for any A,, with P(Bj|Aj) = P(Bj), and vice versa. The 
probability for having both attributes Aj and Bj is 

P(AjnB ( ) = P(Aj) • P(Bj) (84) 

Assuming the variables A and B are independent, the difference between an observed cell count n ; j 
and an expected (based on independence) cell count, 

n • Pj. • P.j = n pL’-lUlJ ( 85 ) 
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should be small. It can be shown that, under the assumption of independence, 


I J 

E E 

i= 1 J = 1 


n ,- n -j 
n 


n i- ‘ n -j 
n 


( 86 ) 


is approximately chi-square distributed. The number of degrees of freedom is (I-1)(J-1). 

Thus, the more the actual values, njj, differ from what would be the expected value if the value for 
row i is independent of being in column j 


n i- • n ~j (87) 

n 

the larger the difference 


n I 

The summation over all such differences would then be large. Consequently, a larger chi-square 
value indicates a high degree of association. Conversely, a chi-square value close to zero indicates 
that the variables are independent. 

The example in the next section demonstrates the use for and computations in a contingency table/ 
chi-square test of association. 

5.5 EXAMPLE: LUBRICATION FLUID PARTICULATE CONCENTRATION AS AN INDI- 
CATION OF TURBINE SHAFT WEAR 

Turbine shaft play in a shuttle component is thought to be associated with the metal particulate 
concentration of the component’s lubrication fluid. In general, analysis of used lubricant can be 
used to detect unusual wear and to predict impending failures. A spectrometer (based on emission/ 
absorption principles) is used to determine the amounts of wear-metals or elements in a lubrication 
fluid sample. 

To ascertain whether lubrication particulate concentration may be used as a performance indication 
of shaft wear, data on shaft wear and particulate concentration were obtained. The following 
contingency table summarizes this data. 
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Shaft Wear 


Particulate Concentration (mg/ml) 



10 -4 <x 

10' 4 <x< I0‘ 3 

1 0~ 3 <x< 10 2 

10 2 <k 10' 1 

x> 10' 1 


Excessive Shaft Wear 

1 

3 

7 

4 

10 


Minor Shaft Wear 

5 

6 

7 

14 

6 


No Shaft Wear 

15 

11 

, 

9 


4 



21 


23 

28 

20 


The “expected” cell counts, assuming there is no association, are: 




4.69 

4.46 

5.13 

6.25 





6.79 

7.8 

9.5 





8.75 

10.06 

12.25 




Carrying out the computation for the measurement of the difference between observed and expected 
values by equation (86), the chi-square value is: 


X 2 = 22.12 * 89 > 


with (3-l)(5-l) = 8 degrees of freedom. Using a chi-square table, the probability for no association is 
less than .5%. Therefore, the metal particulate concentration of lubricant fluid would be a strong 
indication of performance/reliability. The particulate concentration parameter would be a good 
candidate for performance trending of the component’s turbine shaft wear. 
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Chapter 6 

TECHNIQUE SELECTION 


The graphical, descriptive statistics and analytical techniques described in this document should be 
employed to identify and evaluate potentially hazardous or otherwise significant trends. In many 
cases, simple descriptive statistics and graphical representations of the data will suffice in trend 
identification/evaluation. When a numerical measurement of trend is needed, the analytical regres- 
sion and small sample techniques apply. The accompanying flowchart depicts the general progres- 
sion in selection of techniques for a trend analysis. 
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